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Abstract 

Background: Employing waiting list control designs in psychological and behavioral intervention research may 
artificially inflate intervention effect estimates. This exploratory randomized controlled trial tested this proposition in 
a study employing a brief intervention for problem drinkers, one domain of research in which waiting list control 
designs are used. 

Methods: All participants (N = 185) were provided with brief personalized feedback intervention materials after 
being randomly allocated either to be told that they were in the intervention condition and that this was the 
intervention or to be told that they were in the waiting list control condition and that they would receive access to 
the intervention in four weeks with this information provided in the meantime. 

Results: A total of 157 participants (85%) were followed-up after 4 weeks. Between-group differences were found in 
one of four outcomes (proportion within safe drinking guidelines). An interaction was identified between experimental 
manipulation and stage of change at study entry such that participant change was arrested among those more ready 
to change and told they were on the waiting list. 

Conclusions: Trials with waiting list control conditions may overestimate treatment effects, though the extent of any 
such bias appears likely to vary between study populations. Arguably they should only be used where this threat to 
valid inference has been carefully assessed. 

Keywords: Randomized controlled trials. Research methods. Waiting list control design, Alternate explanation. Alcohol, 
Brief intervention 



Background 

There has been growing concern over the use of waiting 
Ust control designs in psychological and behavioral inter- 
vention research [1-8]. While there are ethical advantages 
to a waiting list design because it allows for the provision 
of care (if delayed) to research participants who are seek- 
ing help, whilst permitting a non-intervention evaluation, 
it has been noted that such designs may overestimate 
intervention effects [1-7]. This is because participants 
assigned to a waiting list control condition appear to im- 
prove less (or not at all) than would be expected for 
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people who are concerned about their behavior and who 
are taking steps to change. In a discussion of this possibil- 
ity, Miller and RoUnick point out that examination of pat- 
terns of change in participants assigned to waiting list 
control conditions may indicate that they perceive they 
are expected to 'wait' to change until receiving the inter- 
vention and compliandy do so [9]. This contrasts with 
studies not employing waiting list designs in which control 
group participants tend to improve [10-12]. 

There has been some research on this topic, usually 
framed in the context of expectancies or demand char- 
acteristics, though there has been little dedicated study 
of the latter outside laboratory settings [13]. Previous 
studies have included a range of waiting list designs, 
some in which the participant is told that they will have 
to wait to receive treatment and others where a period 
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of monitoring is described as a necessary baseline 
[14,15]. For the rare interventions where it is impos- 
sible for the participant to know whether they are re- 
ceiving an intervention (e.g., distance healing), research 
has also been conducted where the presence or absence of 
the intervention is crossed with the participant being told 
that they are, or are not, receiving the intervention in 
order to estimate the impact of expectancies [16]. Other 
factorial designs have also been used to evaluate unin- 
tended impacts of the research process [17]. 

Brief interventions for problem (i.e. hazardous or 
harmful) drinking is one area of research utilizing wait- 
ing list control designs [18,19]. Given evidence that 
problem drinking is often resolved without treatment 
[10,20], the use of a waiting list control design may be 
unethical if it has the effects ascribed to it. While other 
aspects of the research process have been evaluated in 
this field [21], there has been no experimental study of 
the effects of waiting list control conditions on partici- 
pant drinking, i.e., of reactivity within this research de- 
sign. In this exploratory randomized controlled trial we 
sought to develop a method for testing two hypotheses: 
Hypothesis 1: When given the same intervention mater- 
ial, people told that they are in the waiting list control 
condition will report heavier drinking at follow-up than 
people told that they are in the intervention condition (a 
main effect); Hypothesis 2: Participants who are more 
ready to change their drinking will report heavier drink- 
ing at follow-up compared to participants who are less 
ready to change (an interaction effect). 

Methods 

This study employed a 'no difference' trial paradigm in 
which all participants are given access to the same inter- 
vention while other aspects of the research process are 
experimentally manipulated [22]. Potential participants 
were recruited through Toronto newspaper advertise- 
ments inviting people 'concerned about their drinking' 
to help with the evaluation of self-directed interventions. 
The newspaper advertisement also mentioned that com- 
pensation would be provided and that the study was not 
a treatment program. Respondents telephoned study 
staff and were mailed out a consent form and baseline 
assessment questionnaire. Participants who returned the 
consent form and baseline questionnaire were randomized 
to be told that they were either in the: a) intervention con- 
dition and sent feedback generated from a known effective 
intervention. Check Your Drinking (CYD) website [23-26]; 
or b) waiting list condition and would be sent details of the 
intervention in 4 weeks and provided with information 
about their drinking in the meantime (which was identical 
to the feedback generated using the CYD website). See 
below for the exact content of this experimental manipula- 
tion. Follow-up was conducted four weeks after baseline 



and participants received $20 for returning the follow-up 
questionnaire. Both baseline and follow-up assessments 
were conducted by postal questionnaire. After the follow- 
up questionnaire was returned, all participants were pro- 
vided access to the Alcohol Help Centre (AHC) online 
program and participants in the 'waiting list control condi- 
tion' were told that this was the intervention [27]. Thus, 
the only difference between the two groups was that those 
in one condition were told that they were receiving an 
intervention and those in the other were told that they had 
to wait four weeks before getting access to the interven- 
tion. The follow-up length of four weeks was partly chosen 
to replicate a published trial of a similar web-based inter- 
vention for problem drinkers which used a waiting list con- 
trol design [18]. 

Text used as experimental manipulation 
Text used in intervention condition: "You are in the 
intervention condition of this study. We have developed 
a personalized feedback intervention for people con- 
cerned about their drinking. We generated a Final Re- 
port for you from this intervention and it is included 
with this letter." 

Text used in waiting list condition: "You are in the 
waiting list condition of this study. You will need to wait 
for 4 weeks until we can send you the intervention ma- 
terials. In the meantime, we have generated some per- 
sonalized information about your drinking and it is 
included with this letter." 

Ethics 

After providing a description of the study to the sub- 
jects, written informed consent was obtained. This con- 
sent procedure and the conduct of the study were 
approved by the standing ethics review committee of the 
Centre for Addiction and Mental Health. 

Baseline and outcome variables and analysis plan 

The outcome variables were: number of drinks in a typ- 
ical recent week, largest number of drinks on one occa- 
sion, and the AUDIT-C [28], the consumption subscale 
of the Alcohol Use Disorders Identification Test AUDIT, 
[29,30], which includes three items (frequency of drink- 
ing, number of drinks per drinking day, and frequency 
of five of more drinks on one occasion) and total scores 
range from 0 to 12. The final outcome measure was un- 
planned prior to the completion of the study and com- 
prised the proportion of participants drinking within the 
Canadian safe drinking guideline (for males, no more 
than 15 drinks per week and three drinks per drinking 
day; for females, no more than 10 drinks per week and 
two drinks per drinking day) [31]. The questionnaire 
contained a graphic describing a standard drink which 
in Canada contains 13.6 grams of ethanol [31]. These 
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variables were assessed at baseline and at follow-up. In 
addition, at baseline, participants provided information 
on their demographic characteristics, and completed the 
other items of the full AUDIT and the Readiness to 
Change (RTC) questionnaire [32,33]. 

Continuous outcome variables were examined for out- 
liers and Winsorized to normalize the distribution by re- 
placing values more than three standard deviations from 
the mean with the next highest value. Analyses for the 
continuous variables were conducted using stepwise linear 
regression in which the baseline value of each outcome 
variable was entered in Step 1. In Step 2, participant ex- 
perimental condition was entered. In addition, the Action 
subscale of the RTC was entered as a main effect continu- 
ous variable. Finally, in Step 3, an interaction term was 
added. The categorical variable, proportion of participants 
drinking within safe drinking guidelines, was compared 
between experimental conditions using Fisher's exact test. 
Note that an earlier version of this analysis was conducted 
using logistic regression in order to allow the inclusion of 
the Action subscale as one of the predictors (including a 
main effect and an interaction term with experimental 
condition). However, as there was no significant {p> .05) 
main effect or interaction effect of the Action subscale, 
and because the inclusion of the Action subscale did not 
substantively influence the main effect of experimental 
condition observed, the simpler Fischer's exact test is pre- 
sented in this paper. Data missing at follow-up were not 
replaced to mimic the treatment of missing data employed 
in studies evaluating brief interventions of this type with 
waiting list control designs [18] and because attrition was 
not judged likely to be problematic''. 

Study participants 

A total of 191 participants responded to the newspaper 
advertisements and returned a signed consent form and 
baseline questionnaire. Of these, 185 were hazardous or 
harmful drinkers (as defined by an AUDIT score of 8 or 
more) and were included in this study. Bivariate compar- 
isons were made on demographic and baseline drinking 
characteristics between experimental condition and there 
were no significant differences {p > .05). The mean age of 
the 185 participants was 47.3 (SD 11.4), 70.3% were male, 
55.1% had some post-secondary education, 31.4% were 
married or living in common law relationships, 52.4% 
were full or part-time employed, and 48.1% reported a 
family income of less than $30,000 per year. Baseline 
levels of problem drinking were quite severe for a 
community-recruited sample with a mean AUDIT score 
of 24.1 (SD 7.0; a score of 20 or more on the AUDIT is in- 
dicative of possible alcohol dependence). Participants re- 
ported typically consuming an average of 35.3 (SD 21.4) 
drinks per week, and the mean highest number of drinks 
consumed on one occasion in the last year was 14.7 (SD 



7.5). Participants' baseline AUDIT-C mean score was 9.0 
(SD 2.0). 

Results 

Follow-up rates were satisfactory with 157 (85%) return- 
ing their four-week survey. An additional two partici- 
pants did not complete the items for the Action subscale 
at baseline, leaving 155 participants available for analysis. 
Figure 1 provides displays an overview of the recruit- 
ment and follow-up rates of the trial. 

There were no significant differences in follow-up 
rates between experimental condition (Told intervention 
condition = 83.5%; Told waiting list = 86.2%; p = .68). 
Table 1 displays the data for the three regression ana- 
lyses. In order to facilitate interpretation of these ana- 
lyses. Table 2 displays the means (SD) of the three 
continuous variables at baseline and follow-up, separated 
by experimental condition and a median split on partici- 
pants' baseline Action subscale scores. For the outcome 
variable, typical drinks per week, there was a significant 
interaction between experimental condition and the Ac- 
tion subscale (p = .05). Inspection of the estimated mar- 
ginal means on Table 2 revealed that for participants 
who rated themselves as low on the Action subscale, 
condition allocation (waiting list or intervention) had lit- 
tle or no impact on their drinking. However, participants 
who scored themselves above the median on the Action 
subscale and who were in the waiting list condition re- 
ported drinking about 6 drinks per week more than their 
counterparts in the intervention condition. A similar 
pattern of results was observed for the largest number of 
drinks consumed on one occasion (p < .01). There were 
no significant experimental condition or interaction ef- 
fects (p > .05) for AUDIT-C scores. 

Proportion drinking within safe drinking guidelines 

Despite scoring 8 or more on the AUDIT, 4.3% of partic- 
ipants [(8/185; or 4.5% (7/157) of participants foUowed- 
up], were drinking within safe drinking guidelines at the 
time of the baseline assessment. At follow-up, 20.4% 
(32/157) were drinking within safe drinking guidelines. 
There was a main effect of study condition, both on the 
proportion of participants drinking within safe drinking 
guidelines at follow-up (Fisher's Exact test, p = .03; Told 
intervention condition = 27.6%; Told waiting list = 13.6%) 
and on the increase in proportion of those who did not 
drink within safe drinking guidelines at baseline but did so 
at follow-up (Fisher's Exact test, p = .009; Told interven- 
tion condition = 25.0%; Told waiting list = 8.6%). 

Discussion 

This exploratory trial developed a research design to 
examine the effects of being in a waiting list control 
condition in psychological and behavioural intervention 
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Assessed for eligibility 
(n= 191) 



Excluded (n= 6) 



Failed to meet inclusion 
criteria; i.e., AUDIT < 8 
(n=e) 



Randomized 
{n= 185) 



Told Intervention, mailed CYD 
materials (n = 91) 



Told Waiting list, mailed CYD 
materials (n =94) 



Lost to follow-up 
4 weeks (n = 15) 

- questionnaires not received 



Lost to follow-up 
4 weeks (n = 1 3) 

- questionnaires not received 



a 
c 
< 



Analyzed (n = 76) 



Analyzed (n = 79) 

- additional 2 participants had 
missing Action scale data 



Figure 1 CONSORT flowchart. 



research. The intention behind this study design was to 
achieve a contrast between the effect of intervention de- 
Uberately confounded with expectancy (as is usually de- 
livered and evaluated), versus the effect of intervention 
without expectancy, thereby estimating the expectancy 



effect as the difference between the two. Specifically, the 
design allows for a test of the impact of being told that 
the participant is in the waiting list control condition be- 
cause all participants are provided with the same interven- 
tion materials (the personalized feedback report generated 



Table 1 Relationship of four-week follow-up drinking with intervention condition and level of action for change intent 
after controlling for baseline drinking 



Drinks per week 



Highest number 



AUDIT-C= 



Predictor 




P (95% C.I) 




P (95% C.I) 




P (95% C.I) 


STEP 1 


.32*** 




.45*** 




.19*** 




Baseline drinking 




0.46 (0.35 - 0.56)*** 




0.50 (041 - 0.59)*** 




0.59 (0.39 - 0.79)*** 


STEP 2 


.05** 




.01 




.06** 




Condition'' 




1.51 (-2.71 - 5.73) 




0.45 (-0.84 - 1 .74) 




0.27 (-0.50 - 1.03) 


Action subscale 




-0.87 (-141 - -0.32)** 




-0.12 (-0.29 - 0.44) 




-1.65 (-0.26 - -0.06)*** 


STEP 3 


.02* 




.03** 




.01 




Condition*Action 




1.08 (0.00- 2.16)* 




0.48 (0.16 - 0.81)** 




0.14 (-0.06 - 0.34) 



Note: *p < .05. **p < .01 . ***p < .001 . 

^AUDIT-C is a composite measure that consists of respondents scores on frequency of drinking, drinks per drinking day, and frequency of five or more drinks on 
one occasion. 

"^Condition: O = told in intervention condition; 1 =told in waiting list condition. 



Cunningham ef al. BMC Medical Research Methodology 201 3, 1 3:1 50 
http://www.biomedcentral.com/1471-2288/13/150 



Page 5 of 7 



Table 2 Mean (SD) drinking variables at baseline and four-week follow-up by study condition (told in intervention 
versus told on waiting list) and median split on Action stage variable (N = 155)^ 



Low action score 
Told intervention (n = 36) Told waiting list (n = 46) 



High action score 
Told intervention (n = 40) Told waiting list (n = 33) 



Typical weekly drinking [Mean (SD)] 

Baseline 32.8(17.7) 
4-week 25.0(17.7) 
Highest number on one occasion [Mean (SD)] 

Baseline 12.9(5.7) 
4-week 8.6 (4.5) 

Audit-C score*" [Mean (SD)] 

Baseline 8.9(1.7) 
4-week 6.9 (2.5) 



334 (20.2) 
24.2 (17.7) 

14.1 (8.4) 
8.5 (5.4) 

8.8 (2.3) 
7.1 (2.7) 



32.6 (16.8) 
145 (9.8) 

15.3 (5.8) 

7.6 (5.3) 

9.2 (1.8) 

5.7 (3.0) 



35.5 (25.3) 
214 (15.5) 

14.1 (8.4) 
8.9 (6.3) 

9.0 (2.0) 

6.1 (2.4) 



^Analyses employed Action scale as a continuous measure (see Table 1). Median split values on this table are provided for descriptive purposes. 

"^AUDIT-C is a composite measure that consists of respondents scores on frequency of drinking, drinks per drinking day, and frequency of five or more drinks on 

one occasion. Scores range from 0-12. 



from the CYD online program). This design is equiva- 
lent to the subtractive expectancy placebo proposed by 
Suedfeld [34] where all participants are given the interven- 
tion but a randomized half is led to expect that it is inert 
with respect to the problem being treated. The advantage 
of the personalized feedback material for this particular 
study was that it was brief and could credibly be described 
as merely information to those in the waiting list control 
condition, while at the same time being a plausible inter- 
vention for those in the other study condition. As such, 
the feedback has intrinsic placebo properties and this 
intervention is not different in this respect from other 
interventions. 

Mixed findings were obtained in relation to Hypoth- 
esis 1, only in the sense that while a lower proportion of 
participants in the control condition reported drinking 
within recommended guidelines at follow-up than their 
intervention group counterparts (an originally unplanned 
analysis), there were no between-group differences in the 
number of drinks per week, highest number of drinks on 
one occasion, and the heaviness of drinking measured 
with the AUDIT-C (the three planned outcome measures). 
The observed differences were in the anticipated direction 
and are generally coherent with an expectation that they 
would be statistically significant in a larger sample. Our 
test of Hypothesis 2 strengthens this possibility as it dem- 
onstrated heavier drinking among waiting-list control par- 
ticipants who were more ready to change compared to 
their counterparts receiving the "intervention." This find- 
ing is consistent with the overarching hypothesised mech- 
anism of effect, that waiting list allocation interrupts 
efforts at change, and also points to the importance of 
consideration of readiness for, or activities towards, 
change in this regard. 

While it would be unjustified to conclude that a wait- 
ing list effect exists on the basis of statistically significant 



main effects from one of the four outcomes, the inter- 
action effects observed here are provocative. For two of 
the continuous outcome variables, largest number of 
drinks on one occasion, and number of drinks in a typ- 
ical week, there were interactions between experimental 
condition and readiness to change (started to do some- 
thing about their drinking on entering the trial). It 
should be noted that these analyses were conducted in 
two different ways - the first using the categorical stages 
of change designation calculated using the full RTC scale 
[32] and the second employing just the action subscale. 
This repetition of analyses is justified given the explora- 
tory nature of this trial, though the lack of an a priori 
constructed data analysis plan is acknowledged. Also, 
the finding that only the action subscale was predictive 
of outcome aligns with findings from previous research 
on Stages of Change albeit using a different measure of 
the construct [35]. 

Despite the strengths in the employed research design, 
one limitation may also be implicit in it. An additive 
model is assumed in that both groups are intended to 
actually read the feedback report [36]. Any differences 
between groups in doing so, will serve to inflate appar- 
ent waiting list effects entirely due to expectancies. Spe- 
cifically, if being told that being in the intervention 
condition makes one more likely to read, or to think 
about, the intervention material, then the nature of the 
effect involves an interaction and is thus more complex 
[36]. This means that the effects observed here are con- 
tingent upon this specific feature of this study. The ex- 
ploratory nature of this study also imposes various limits 
to inference. A power calculation was not undertaken a 
priori, making both the attainment of statistical signifi- 
cance here less important, and relatedly, clear and unam- 
biguous interpretation of study findings more challenging. 
Effect estimates for future larger replication studies are 
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now available in this study [37]. Another limitation was 
that there was no way to distinguish between those who 
were responding to the newspaper advertisements because 
they were looking for help regarding their drinking and 
those more motivated by the $20 for participating in the 
trial. To the extent that the financial incentive motivated 
participation it is likely that the observed differences 
under- represent the size of the true effects, particularly in 
light of the observed readiness to change findings. Alter- 
natively, if the newspaper recruitment resulted in a sample 
that was highly motivated to change (e.g., in comparison 
to a proactively recruited sample) then the results of this 
trial could overestimate the impact of a waiting list design 
in such a population. The latter possibility is highly un- 
likely in treatment studies for help seekers, but should be 
borne in mind for brief intervention trials based on oppor- 
tunistic recruitment in healthcare settings. 

Future directions for this research include examining 
whether a waiting list control manipulation has more 
impact in particular research settings and with specific 
populations. For example, in situations where the ma- 
nipulation is delivered face-to-face, more reactivity rela- 
tive to the waiting list control condition may result. 
There is obvious value in examining the mechanisms be- 
hind the hypothesised negative impact of the waiting list 
control condition. This is particularly true for study pop- 
ulations of confirmed help-seekers. Qualitative interviews 
could also be used to investigate negative reactions of par- 
ticipants assigned to waiting list or other types of control 
conditions, particularly among those with clear prefer- 
ences. Separating true expectancy effects associated with 
compliance with demand characteristics implicit in wait- 
ing list study conditions from participants becoming irri- 
tated or disconsolate over not getting the help they hoped 
to receive and reduce their own efforts to drink less 
(termed resentful demoralization [38]) will be a further 
challenge to address when this field of investigation is 
more developed. 

Conclusions 

The results of this exploratory study give further weight 
to the generally increasing levels of scrutiny recently 
given to control conditions, and for the interpretation of 
findings from trials employing wait-list control designs 
in particular. Further, these results point to the need for 
caution regarding the ethics of assigning participants ac- 
tively ready to change to a waiting list control condition. 

Endnote 

"At the suggestion of one of the reviewers, the primary 
analyses were re-conducted using an intention-to-treat 
approach (missing data at follow-up replaced with the 
respective baseline values). However, as the pattern of 



results was unchanged from that reported here, this al- 
ternative analysis was not reported in this paper. 
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